Over the past decade there has been exponential growth in the prevalence of streaming media. Users increasingly view television shows, movies, and video clips through websites and video streaming services such as Hulu, Netflix, and YouTube. Interactive forms of streaming media are also available. For example, some videos allow a user to affect video playback in real time by interactively selecting a sequence of video segments to play. These interactive videos do not, however, enable the user to alter only a portion of a video frame. Rather, the user is only able to change the contents of the video by causing a different video segment to be played. Consequently, there exists a need for systems and methods for interacting with and altering video frame portions in real-time.